Nowcasting the mood of the nation

Have you ever had the impression that everyone around you is stressed? Or is it just you? Answering a simple question like this can be very hard, as it would involve interviewing a large sample of people, and asking the right questions, in order to assess their levels of stress (unless you want to measure cortisol levels in faecal samples, as done in wild animal populations [1]). The truth is that this is just one of many aspects of a population that are very difficult to measure or even detect. We often talk about macroeconomic quantities (such as the current level of inflation, or economic growth), forgetting that these quantities always refer to the past, since it takes several months to collect, aggregate and analyse the various economic indicators. Measuring the state of a society or an economy in real time is not an easy task. It is a task that certain practitioners call " nowcasting " [2]. Our group has long been interested in nowcasting certain aspects of society by using the contents of social media. Active Twitter users in the UK number more than 10 million, Facebook users more than 31 million. Twitter, in particular and very helpfully, encourages its 200 million users worldwide to make their posts, commonly known as tweets, publicly available and tagged with the users location. Tweets have another advantage: they tend to be of-the-moment, sent on impulse. They have immediacy; they reflect what the sender is feeling at the time, not what he or she feels looking back, a considered opinion from later. All this generates a massive stream of status update and a new wave of experimentation and research. Can we find out something about Twitter and Facebook users-as a population-by analysing those posts? Our first project back in 2010 had involved measuring the prevalence of flu-like symptoms in the general UK population, based on the contents of Twitter. We set up a system to read a large amount of tweets from the 54 largest cities in the UK, and a pipeline for the statistical analysis of text. We also collected the actual flu rates from the Health Protection Agency (HPA), so that we could use supervised computer-learning algorithms to map textual content to flu levels. We could have fed in key words and phrases, such as 'flu', or 'off work', or 'home in bed' and asked our text-recognition software to search …